Skip to main content

7.1 - Setting up TensorBoard

Training a reinforcement learning agent is an empirical process. To understand if the agent is learning, improving, or stagnating, you must visualize its performance metrics over time. TensorBoard is the industry-standard tool for this purpose.

This section provides the standard operating procedure for instrumenting your stable-baselines3 training runs to generate and view TensorBoard logs.

The Role of TensorBoard​

TensorBoard runs as a local web server that parses log files generated during training and displays them as interactive graphs in your web browser. This allows you to monitor key metrics in real-time and analyze the agent's learning trajectory.

Instrumentation and Visualization Workflow​

The process involves two parallel tasks: running the training script to generate logs and running the TensorBoard server to consume them.

       +------------------------------------+
       |          Terminal 1                |
       |  (Training Process)                |
       +------------------------------------+
                      |
                      | 1. python train.py
                      |
                      v
       +------------------------------------+
       |      stable-baselines3 PPO         |------> Generates Logs
       +------------------------------------+

       +------------------------------------+      +-----------------------+
       |          Terminal 2                |      |     Web Browser       |
       |  (Visualization Process)           |      +-----------------------+
       +------------------------------------+                ^
                      |                                      |
                      | 2. tensorboard --logdir ...          | 3. View at
                      |                                      | localhost:6006
                      v                                      |
       +------------------------------------+                |
       |      TensorBoard Web Server        |<------ Reads Logs
       +------------------------------------+

Setup and Execution​

  • Step 1: Instrument the Training Script This step is already complete in our train.py script. The tensorboard_log parameter directs stable-baselines3 to write logs to the specified directory.

    In train.py:

    model = PPO(
        # ... other params
        tensorboard_log="./sc2_rl_tensorboard/"
    )

  • Step 2: Run the Training Process In your first terminal, with your virtual environment active, start the training as usual.

    python train.py

    This will create a sc2_rl_tensorboard/ directory in your project folder and begin populating it with log files.

  • Step 3: Launch the TensorBoard Server

    1. Open a second, separate terminal window.
    2. Navigate to the same project directory.
    3. Activate the same virtual environment. This is a critical step, as it ensures the tensorboard command is available. 
      # Windows
      .\venv\Scripts\activate
      # macOS / Linux
      source venv/bin/activate
    4. Launch the server, pointing it at the log directory. 
      tensorboard --logdir ./sc2_rl_tensorboard/

Expected Outcome​

After running the launch command, your second terminal will display a message similar to this:

TensorBoard 2.15.2 at http://localhost:6006/ (Press CTRL+C to quit)

Navigating to http://localhost:6006/ in your web browser will open the TensorBoard dashboard. As your agent trains in the first terminal, you can refresh this dashboard to see the graphs update with new data, providing a live view of the learning process. In the next section, we will cover which of these graphs are most important to watch.